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Abstract 

The  problem  of  estimating  the  parameters  of  multiple  independent  continuous-time  Markov  on- 
off  processes  is  considered.  The  objective  is  to  minimize  the  total  mean  square  error  (MSE)  under  a 
constraint  on  the  total  sensing  time.  The  Fisher  information  matrix  for  the  primary  traffic  model  and 
the  maximum  likelihood  estimator  are  obtained.  A  sequential  estimation  strategy  is  proposed  which 
operates  under  an  epoch  structure  with  growing  epoch  length.  It  is  shown  that  this  sequential  estimation 
strategy  is  asymptotically  efficient  as  the  total  sensing  time  increases.  This  result  finds  application  in 
opportunistic  spectrum  access  where  secondary  users  need  to  estimate  the  channel  occupancy  model  of 
the  primary  system  for  efficient  exploitation  of  spectrum  opportunities. 

Index  Terms 

Channel  estimation,  continuous-time  Markov  process,  sequential  estimation,  cognitive  radio,  oppor¬ 
tunistic  spectrum  access  (OSA). 


I.  Introduction 

In  opportunistic  spectrum  access,  secondary  users  sense  and  access  temporally  unused  channels 
in  the  spectrum  without  causing  unacceptable  interference  to  primary  users  [1].  An  accurate 
stochastic  modeling  of  the  primary  system  channel  occupancy  plays  a  crucial  role  in  designing 
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the  optimal  algorithms  for  sensing,  traeking,  and  exploiting  speetrum  opportunities.  For  instanee, 
in  [2]-[4],  the  ehannel  oeeupaney  of  the  primary  system  is  modeled  as  a  eontinuous-time  Markov 
on-off  proeess.  Given  these  parameters,  optimal  sensing  and  aeeess  strategies  of  the  seeondary 
users  are  designed.  In  praetiee,  however,  the  stoehastie  model  of  the  primary  traffie  may  not  be 
known  a  priori',  sueh  a  model  must  be  learned  through  ehannel  sensing. 

In  this  paper,  we  eonsider  the  problem  of  estimating  the  parameters  of  multiple  independent 
eontinuous-time  Markov  on-off  proeesses.  The  objeetive  is  to  minimize  the  total  mean  squared 
error  (MSB)  aeross  all  ehannels  under  a  eonstraint  on  the  total  sensing  time.  To  this  end,  we 
obtain  the  Fisher  information  matrix  and  the  maximum  likelihood  estimator  (MLE).  Given  that 
the  optimal  alloeation  of  the  total  sensing  time  to  multiple  ehannels  depends  on  the  unknown 
parameters,  a  sequential  estimation  strategy  is  proposed  whieh  dynamieally  adjusts  the  alloeation 
of  sensing  time  based  on  the  partial  learning  results  obtained  up  to  the  eurrent  time.  Referred 
to  as  SEINE  (Sequenee  Estimation  with  Inereasing  Nested  Epoehs),  the  proposed  sequential 
estimation  poliey  operates  under  an  epoeh  strueture.  Within  eaeh  epoeh,  ehannels  are  sensed  in 
turn,  eaeh  for  a  fraetion  of  the  epoeh  length  with  the  fraetion  determined  by  the  eurrent  estimate 
of  the  ehannel  parameters.  The  epoeh  length  grows  over  time  to  take  advantage  of  the  inereasing 
aeeuraey  of  the  estimates.  It  is  shown  that  SEINE  is  asymptotieally  effieient,  i.e.,  it  aehieves  the 
Cramer-Rao  Bound  (CRB)  as  the  total  sensing  time  grows. 

Beaming  the  stoehastie  models  of  primary  ehannel  oeeupaney  has  reeeived  relatively  little 
attention.  There  exist  a  few  published  results,  all  foeusing  on  a  single  ehannel  and  diserete 
sampling.  For  example,  in  [5],  [6],  Maximum  Eikelihood  and  Bayesian  estimation  of  ehannel 
parameters  under  a  uniform  sampling  strategy  were  studied.  In  [7],  relationship  between  estima¬ 
tion  aeouraey,  the  number  of  samples  taken,  and  the  ehannel  state  transition  probabilities  was 
analyzed  by  using  the  sampling  and  estimation  framework  proposed  in  [5].  Park  et  al.  in  [8] 
proposed  a  ehannel  state  predietor  based  on  the  reinforeement  learning  teehniques  where  the 
ehannel  model  is  assume  to  be  a  hidden  Markov  proeess.  In  [9],  a  wavelet  transform  based 
ehannel  estimator  was  proposed.  In  [10],  the  performanee  of  the  single  ehannel  MEE  of  the 
uniform  and  random  diserete-time  sampling  strategies  were  eompared.  It  is  demonstrated  that 
when  the  samples  are  sparse  enough,  the  random  sampling  outperforms  the  uniform  sampling. 
The  analysis  of  [10]  assumes  that  the  utilization  faetor  of  the  ehannel  is  known  whieh  reduees  the 
problem  to  a  single  (sealer)  parameter  estimation  problem.  A  dynamie  programming  approaeh 
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is  proposed  to  obtain  the  best  and  the  worst  sampling  seheme  whieh  ean  be  solved  numerieally. 
For  the  time-varying  ehannel  parameters,  an  adaptive  random  sensing  seheme  is  proposed  and 
shown  to  outperform  its  counterpart  using  uniform  sensing. 

II.  Problem  Statement  and  Fundamental  statistics 

Consider  a  network  that  consists  of  M  channels.  These  M  channels  are  licensed  to  an  unslotted 
primary  network.  The  spectrum  occupancy  of  channel  m  is  modeled  as  a  continuous-time  Markov 
process  with  two  states:  Sm{t)  =  1  (busy)  and  S'm(f)  =  0  (idle).  These  M  Markov  processes 
are  jointly  independent.  In  particular,  for  channel  m,  the  sojourn  times  in  the  busy  and  idle 
states  are  exponentially  distributed  with  rates  Ai,m  and  Ao,m,  respectively.  Theses  parameters  are 
unknown  to  the  secondary  system.  A  secondary  user’s  objective  is  to  learn  the  primary  network 
occupancy  model.  It  aims  to  estimate  the  set  of  channel  parameters  {Ao,m,  sensing 

these  M  channels. 


BUSY 


BUSY 


BUSY 


T. 


TIME 


Fig.  1.  Channel  Sensing  Model 


We  assume  that  the  secondary  user  can  only  sense  one  channel  at  a  time,  and  there  is  a 
budget  for  the  total  sensing  time.  An  illustration  of  a  particular  sensing  scenario  is  shown  in 
Fig  1  where  the  secondary  user  monitors  a  particular  channel  continuously  for  a  period  of  time 
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before  switehing  to  a  different  ehannel.  It  is  intuitive  that  a  ehannel  with  greater  statistieal 
varianee  requires  longer  total  sensing  time  to  aehieve  the  same  level  of  aeouraey  as  that  for 
a  ehannel  with  smaller  varianee.  The  main  ehallenge  here  is  to  design  a  sensing  poliey  that 
alloeates  time  spent  on  eaeh  ehannel  optimally. 

While  the  eontinuous-time  on-off  Markovian  abstraetion  of  the  primary  ehannels  is  widely 
aeeepted,  it  has  several  subtle  modeling  eomplieations.  Speeifically,  the  transitions  of  the  primary 
traffie  are  instantaneous.  From  a  signal  theoretie  point  of  view,  such  a  process  has  infinite 
bandwidth  and  no  discrete-time  sensing  can  be  made  without  loss  of  information.  In  this  paper, 
we  adopt  a  continuous  sensing  model  where  it  is  assumed  that  the  secondary  user  can  observe 
the  channel  continuously.  This  of  course  can  only  be  an  approximation  of  a  practical  sensing 
mechanism,  but  it  has  the  significant  theoretical  benefit  that  the  sensing  process  does  not  lead 
to  information  loss. 

We  also  have  to  make  a  few  additional  assumptions  on  sensing.  The  on-off  model  requires 
a  hypothesis  test  to  decide  whether  a  particular  channel  is  idle  or  busy.  When  the  observation 
is  noisy,  such  hypothesis  testing  suffers  from  miss  detection  and  false  alarm.  The  modeling  of 
sensing  errors  leads  to  significant  complications  and  is  not  considered  in  this  paper.  In  addition, 
switching  from  one  channel  to  another  also  takes  time  in  practice.  This  too  will  be  ignored  in 
our  development. 

A.  Likelihood  Function,  Sufficient  Statistics,  and  Fisher  Information:  A  Single  Interval  Analysis 

In  this  section  we  focus  on  a  single  interval  sensed  from  a  given  channel  and  derive  fun¬ 
damental  statistics.  These  statistics  are  extended  to  the  multi-interval  multi-channel  case  in  the 
next  section.  In  the  following,  the  channel  index  is  dropped  for  the  ease  of  presentation. 

The  information  observed  in  [ti,  ti-|-Ti]  from  a  given  channel  is  illustrated  in  Fig.  2;  it  consists 
of  the  state  St^  of  the  channel  at  the  beginning  of  the  sensed  interval  and  the  realizations  of 
the  consecutive  observed  idle/busy  periods  where  n  is  the  number  of  such  periods. 

Hence  the  observation  model  for  the  interval  [fi,  ti  +  Tf  will  be  {St^,  Denote  by  Zfs 

the  exponential  random  variables  where  ZiS  are  respective  realizations  of  what  the  secondary 
user  has  observed.  From  Fig.  2,  we  have  Zi  >  zi,  Z^  >  Zn,  and  Zi  =  Zi  for  i  =  2, ...  ,n  —  1. 
In  the  following  we  refer  to  zfs  with  f  G  {2, . . . ,  n  —  1}  as  complete  periods  since  Zi  =  Zi  for 
them  which  is  not  true  for  f  =  1,  n. 
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Fig.  2.  Observation  Model 


Now  assume  that  a  realization  of  observation  for  the  interval  +  Ti]  is  given  to  be 

{S'ij  =  s,  Zi  >  zi,  (Zi  =  Zi)^~2,  Zn  >  Znj.  Since  the  state  of  the  channel  during  Zi  is  given  to 
be  s,  the  state  of  channel  during  Z^  will  be  Sj  =  s  +  )(1  —  2s).  Therefore  the  likelihood 

function  given  that  idle/busy  periods  for  the  given  channel  are  exponentially  distributed  with 
parameters  Aq,  and  Ai  is 


/(s,  (2:i)r=l|-^0,  Al)  =  PT{Sto  =  -SlAo,  Ai}  Pr{Zi  >  Zi,  Zn  >  Zn}fz2,...,Zn-i{z2,  •  •  •  ,  ^n-l|Ao,  Ai,  Sto  =  s} 


Aq  +  Ai  Aq  +  Al 

Let  no,  and  ni  be  the  number  of  complete  idle  and  busy  states  (note  that  i  =  1,  n  are  excluded) 
respectively.  Thus  we  have 

(|  —  1)+  n  is  even 


Us  =  { 


n  is  odd. 


ni_s  =  {n-2-ns 

Therefore  the  likelihood  function  can  be  rewritten  as 

Ai- 


/(s,(z.)ILi|Ao,Ai)  = 


L5J 


'As  +  Ai_s 

The  likelihood  function  in  (3)  shows  that  the  sufficient  statistics  for  a  single  interval  is 

L=±iJ  LfJ 
(S,n,  ^  Z2i-l,^Z2i). 


(1) 

(2) 

(3) 


2  =  1 


2  =  1 


(4) 
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The  Fisher  information  is  the  negative  of  the  expeetation  of  the  seeond  derivative  of  the  log 
likelihood  funetion,  L{\q,  Ai)  =  log  /(s,  (2:i)”=i|Ao,  Ai),  with  respeet  to  the  unknown  parameters, 
namely, 

ti+Ti]  =  Ai)) 


where 


-vLA.-t(v,  Ai)  = 


AO  +  l{s  =  l}  1 

- P - 


Ag  (Ao+Ai)2  (Ao+Ai)2 


(Ao+Ai)2  A^  (Ao+Ai)2 

where  ls=j  =  1  if  s  =  j  and  zero  otherwise.  The  Fisher  information  ean  be  expressed  in  elosed 
form  given  in  the  following  Theorem. 

Theorem  1:  The  Fisher  information  matrix  for  a  single  ehannel  whieh  is  sensed  for  a  eontinuous- 
time  interval  [ti,  ti  +  T]  ean  be  written  as, 

T  /I  I  ^/rrw  1 


(5) 


'[ii,  ti+T] 


(Ao+Ai)2 

Proof:  See  the  Appendix  A. 

From  theorem  1  for  the  inverse  of  the  Fisher  information  matrix  we  get, 

o(T) 


r-l 


[ti , 


T  (l  +  o(T)) 
o{T) 


(6) 


B.  Likelihood  Function  and  Fisher  Information  for  Multiple  Intervals 

Sinee  the  ehannels  are  independent,  the  likelihood  funetion  of  the  network  will  be  the  produet 
of  the  likelihood  funetions  of  eaeh  ehannel.  Similarly,  the  Fisher  information  of  the  network  will 
be  the  sum  of  the  Fisher  information  matriees  of  individual  ehannels.  Therefore  without  loss 
of  generality,  we  derive  the  likelihood  funetion  and  the  Fisher  information  matrix  for  multiple 
intervals  for  a  given  ehannel  i. 

Assume  that  the  seeondary  user  has  sensed  K  intervals  +  Tf\}f^^  where  the  jth  and 

(j  +  l)th  intervals  are  separated  by  Atj  =  tj+i  —  {tj  +  T,).  Fig.  3  shows  an  example  where 
K  =  A. 

Using  the  same  notation  introdueed  in  seetion  II-A  and  adding  index  j  indieating  the  jth 
interval,  the  likelihood  funetion  for  K  intervals  is  f{{sj,  Xi^i).  Based  on  the 
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Fig.  3.  Multiple  Intervals 


At, 


1 — r 

^4  ti  +  Ti 


Markov  property  the  likelihood  funetion  ean  be  redueed  to  (derivations  for  two  intervals  are 
given  in  Appendix  B), 


iPr{5't^.i i  “  '®i+i 


Pr{5't^-+i  =  Sj+i} 


(7) 


Denote  the  eontinuous  and  diserete  parts  of  the  likelihood  function  by 


~  f  {^ji  i^h,j)hLl\^0,ii 


Lf  = 


Pr{^t^+i  =  Sj+i} 


The  likelihood  function  can  be  written  as, 


/({^7,(^/.,7)?=i}f=i|Ao.,Ai,)  =nf=,  [L^]  nf^-/  [Lf]. 


(8) 


Note  that  Lf  (stands  for  discrete  likelihood)  only  depends  on  the  state  of  the  channels  at  two 
points  tj+Tj  (end  point  of  the  jth  interval)  and  (the  starting  point  of  the  (j  +  l)th  interval). 
Therefore  the  term  Lf  can  be  regarded  as  the  factor  in  the  likelihood  function  coming  from  the 
discrete  sampling  at  single  points  tj  +  Tj,  and  tj.  Lf  (stands  for  continuous  likelihood)  is  the 
likelihood  function  for  the  single  interval  [tj,  tj  +  Tj]  (computed  in  section  II- A). 

Therefore  the  likelihood  function  for  multiple  intervals  consists  of  two  type  of  factors,  the 
likelihood  functions  of  individual  intervals  (Lf's)  and  the  factors  from  the  discrete  sampling 
(Lf"s)  at  the  end  points  of  the  neighboring  intervals.  Fig.  4  shows  continuous  sampling  (in¬ 
dividual  intervals)  and  discrete  sampling  for  two  intervals.  The  conditional  probability  can  be 
written  as 


Pr{^tj+i  =  Sj+i\St  j+Tj  -  Snjj}  - 


Ai- 


^7+1  >' 


Asj+i,i  +  Ai_ 


+  (-l) 


Ai-s 


Sj  +  1,2 


+  Ai_s 


BUSY 


BUSY 


IDLE 


Continuous  Sampling 


Discrete  Sampling 


IDLE 


Continuous  Sampling 


Fig.  4.  Continuous  and  Discrete  Sampling 


We  also  know 

=  sj+i} 

Therefore  the  likelihood  funetion  is 

=  njliK 


Ai- 


•^1— Sj,i 


X  ii,=i  [ 


I  { _ iWn,',j+«j+i _ ^  ’ 


■’r  ^  ^l  —  Sj  —  i  ^2h, 


_g— (Ao,i+Ai_i)  Aij 


Ai_, 


j+i’" 


j+l’' 


The  Fisher  information  is  derived  from  the  logarithm  of  the  likelihood  funetion.  Therefore  by 
the  same  argument  given  for  (8),  the  Fisher  information  of  the  multiple  intervals  will  be  the  sum 
of  the  Fisher  information  of  the  individual  intervals  plus  the  sum  of  the  information  obtained 
from  the  diserete  samplings  at  the  end  points  of  neighboring  intervals  ,tj  +  T/s  and  fj+i’s.  Here 
we  have  dropped  the  ehannel  index  i  for  the  ease  of  presentation. 


K 


K-1 


tj+Tj]  +  ^ 


i=i 


i=i 


where 


(9) 


(10) 


=  sj+i} 

is  the  information  obtained  from  the  diserete  sampling  at  tj  +  Tj  and  tj+i  whieh  is  a  funetion 
of  Atj  (and  also  the  unknown  parameters  Ao,i,  and  Ai^i). 
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In  the  following  theorem  we  show  the  optimality  of  single  interval  versus  multiple  intervals 
with  the  same  total  length  under  a  sparsity  condition. 

Theorem  2:  There  exists  a  finite  value  r  >  0  such  that  the  Fisher  information  of  a  single 
interval  [to,  h  +  T]  is  larger  than  the  Fisher  information  of  K  intervals  {[tj,  tj  where 

Atj  >  r  Vj  e  {1, . . . ,  iF  -  1},  and  Tj  =  T,  i.e., 

I[to,  to+T]  >  tj+Tj]-  (11) 

Proof:  See  Appendix  B.  ■ 

Therefore  if  the  distance  between  intervals  (At/s)  is  quite  large  (bigger  than  a  constant  r), 
sensing  one  big  interval  with  the  length  T  =  higher  Fisher  information.  Based  on 

this  theorem,  a  sensing  policy  is  called  sparse  if  it  satisfies  the  condition  in  Theorem  2,  namely, 

Atj  >  r. 


III.  Maximum  Likelihood  and  Moment  Estimators 


A.  The  Maximum  Likelihood  Estimator 

The  MLE  (maximum  likelihood  estimator)  is  obtained  by  ^  =  0,  j  =  s,l  —  s,  where 

L  =  log  f{{sj,  {zh,j)lLi}f=iWi,  Ai,i) 


is  the  log  likelihood  function,  and  f{{sj,  Ai,*)  is  given  in  section  II-B.  It  is 

apparent  that  the  MEE  depends  on  the  number  of  intervals  K  and  also  Af/s  and  maximizing 
the  likelihood  function  does  not  lead  to  a  convenient  form  that  can  be  used  as  a  framework  for 
sensing  policy  design.  It  is  worth  noting,  however,  that  if  there  is  only  a  single  period  (K  =  1), 
a  case  that  can  be  optimal  for  some  cases,  the  ME  estimator  can  be  obtained  in  a  closed-form 
as  shown  in  the  following  Eemma. 

Lemma  1:  When  there  is  only  a  single  period  (K  =  1),  the  MEE  can  be  obtained  in  a  closed 
from  (3)  in  section  II-A  to  be. 


Ai— s 


C 


+  rii-s), 


=  CXi.s, 


A 


where  C  =  \/ +  B  —  A,  and 
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A  =  -(1 


l^i=l  ^2i 
Z^i=i  Z2i-1 


)(1 


1  +  ni_. 


B  = 


v^LfJ 

2^i=l  ^2i 


n. 


eIx^  ^2.-1  ^ 

The  MLE  is  simply  obtained  by  putting  the  derivatives  of  the  logarithm  of  (3)  with  respect  to 
As,  and  Ai_s  equal  to  zero  and  solving  two  equations  with  two  unknowns. 


B.  A  Moment  Estimator 


We  propose  a  simple  heuristic  estimator  and  we  prove  that  it  is  asymptotically  efficient.  This 
heuristic  estimator  is  based  on  the  sample  mean  of  the  complete  idle  and  busy  periods  and  is 
independent  of  the  sensing  structure.  It  means  that  the  definition  of  the  estimator  does  not  depend 
on  how  many  intervals  (one  single  interval  or  many)  are  sensed  on  each  channel.  Denote  by 
n\{t),  and  the  number  of  complete  idle  and  busy  periods  sensed  on  channel  i  up  to  the 
time  instant  t  (which  includes  the  observed  periods  from  all  time  intervals  up  to  time  t),  and  let 
zlj,  and  be  the  the  duration  of  jth  such  periods.  Then  the  estimators  are 


= 


n*(t) 


n\{t)  -  1 


(12) 


(13) 


Note  that  the  numerators  of  the  estimators  are  n\{t)  —  1,  and  n\{t)  —  1  instead  of  n\{t)  and  n\{t). 
This  is  to  make  sure  that  these  estimators  are  unbiased. 

When  a  limited  period  of  time  T  is  given  to  the  secondary  user,  it  wants  to  optimally  distribute 
it  among  the  channels  to  minimize  the  total  MSE  (the  sum  of  the  MSE  of  all  parameters).  In 
the  following  we  show  that  the  heuristic  estimators  given  in  (12),  and  (13)  can  achieve  optimal 
efficiency. 


IV.  Sequential  Policy  and  Asymptotic  Properties 
A.  Optimal  Sensing  Policy 

In  this  section  we  consider  an  optimal  sensing  policy  that  distributes  total  sensing  budget  to 
a  set  of  M  channels.  A  sensing  policy  tt  is  defined  by  T'^  =  {Tj^}  where  is  the  jth  sensing 
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interval  on  channel  i  under  policy  tt.  Let  =  Yhj  T^j  and  the  total  sensing  time  is  given  by 

T=  |T-|  = 

We  first  consider  the  problem  of  allocating  the  total  amount  of  sensing  time  to  different 
channels  in  a  single  installment,  i.e.,  we  consider  the  class  of  policies  IIi  where  each  channel 
has  a  single  interval  of  duration  T*.  From  (6),  we  have 

r  +  o(T,) 

[  o(T,)  +  ' 

Since  channels  are  jointly  independent,  we  have 


Since  Cramer-Rao  bound  (CRB)  is  a  lower  bound  on  the  MSB,  first  we  minimize  the  lower 
bound  (Cramer-Rao  Bound),  then  later  we  show  that  we  can  achieve  the  optimal  lower  bound. 
The  total  variance  (Cramer-Rao  bound)  is  given  by 

^(Ag,.  +  Af,)(^  +  ^) 

1 

1=1 

The  problem  is  to  minimize  (14)  under  the  condition 


This  problem  can  be  solved  by  Lagrangian  multiplier  which  gives  the  following  solution,  VI  < 
<  M, 


(^o,i  +  Ki)  +  A)-)  \/(^L  +  ^L)(Ai7  +  ^) 


^0,j  ^0,j 


which  results  in 


Ti  =  air, 


where 


+  at-) 


E£,./(Ag.  +  AU(yi-  +  ^) 
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Therefore  the  optimal  (minimum)  CRB  for  all  polieies  tt  G  IIi  where  eaeh  channel  has  a  single 
interval  is 


M 


min  CRB(T^)  =  CRB*(T)  =  -  V  ^ 


2=1 


Oii 


(18) 


Here  we  denote  by  CRB*(T)  the  optimal  CRB  when  T  =  \T^\  under  the  class  of  policies  Hi 
with  single  installment  per  channel. 

Derivations  above  show  that  the  optimal  time  distribution  among  the  channels  depends  on 
the  values  of  the  unknown  parameters  {Aq,*,  Since  channel  parameters  are  unknown,  to 

achieve  asymptotic  optimality,  we  consider  the  class  of  sequential  policies  that  involves  multiple 
rounds  of  estimation,  each  round  is  based  on  data  collected  up  to  the  previous  round.  For 
example,  within  each  round,  the  sensing  time  spent  on  each  channel  can  be  allocated  based  on 
the  most  current  channel  estimates  according  to  (17).  Note  that  the  use  of  data  in  sensing  makes 
the  sensing  policy  random,  which  complicates  the  MSB  analysis  presented  in  Section  IV-E. 


B.  SEINE:  Sequential  Estimation  with  Increasing  Nested  Epochs 

We  describe  here  policy  SEINE — an  adaptive  estimation  scheme  that  adjust  data  collection 
strategies  based  on  current  estimates.  In  Section  IV-E,  we  will  establish  the  asymptotic  optimality 
of  SEINE.  The  key  of  SEINE  is  to  involve  multiple  rounds  with  increasing  duration.  Within  each 
round,  sensing  time  of  each  channel  is  allocated  according  to  (17)  based  on  the  estimated  channel 
parameters  using  all  data  collected  up  to  the  previous  round. 

Policy  SEINE  (Sequence  Estimation  with  Increasing  Nested  Epochs): 

(1)  Round  0:  Pick  a  constant  cq  =  C  large  enough  with  respect  to  the  average  idle  busy  times 
of  all  channels.  Sense  all  channels  equally  by  ^  time. 

(2)  Round  n  (n  >  1):  Use  all  the  observations  from  rounds  0, . . . ,  n  —  1  to  obtain  the  channel 

parameter  estimates  {Ao,i(n),  Ai^i(n)}^]^  using  (12)  and  (13)  where  t  =  Yl'jZoCj.  Substitute 
these  estimated  parameters  in  (17)  to  get  the  estimates  Q;i(n)s  to  distribute  time  among  channels. 
Pick  the  total  round  time  c„  and  spend  Q;j(n)c„  amount  of  time  on  channel  i,  where  is 

an  increasing  sequence  such  that  lim„^oo  '  =  oo. 

In  what  follows  we  consider  three  asymptotic  properties  of  the  estimator  and  policy  proposed 


above. 
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C.  Asymptotic  Statistics  Properties:  Strong  Consistency 

A  sequence  of  estimators  A(T)  for  parameter  A  is  strongly  consistent  if  A(T)  converges  almost 
surely  to  A. 

A(T)  A  a.s.  (19) 

The  first  step  to  show  consistency  is  to  make  sure  the  number  of  samples  required  for  the 
estimators  in  (12)  and  (13)  tends  to  infinity  as  time  grows.  This  fact  is  shown  in  the  following 
lemma  which  will  be  used  later  too. 

Lemma  2:  Define  the  event 

El={m  the  nth  round,  at  least  one  complete  idle  busy  period  is  sensed  on  channel  i} 

,  then 

Pr{£;;  i.o.}  =  1. 

Proof:  Let  7^  =  min{Ao,i,  Ai^j},  then 

>  FErlangiCn,  4,  7,)  =  1  -  (20) 

fc=0 

where  FEriang{x^A^yi)  is  the  CDF  of  the  Erlang  distribution  of  order  4  with  parameter  7*. 
Based  on  the  definition  of  the  policy  SEINE, 

lim  Cn  =  00.  (21) 

n—^oo 

Therefore 

lim  Pr{£;;}  =  1,  (22) 

n—^oo 

so  that  we  can  conclude 

Pr{Fl^  i.o.}  =  Pr{lim  sup  >  lim  Pi{Ff}  =  1.  (23) 

n—^oo 


Theorem  3:  The  estimators  defined  in  (12)  and  (13)  are  consistent. 
Proof:  Based  on  lemma  2,  we  know 

lim  [Ao,i(T),  Ai,i(T)]  =  [  lim  Ao,i(no),  lim  Ai,i(ni)]. 

T— >00  no^oo  ni^oo 


(24) 
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Since  the  complete  periods  are  i.i.d  random  variables,  based  on  the  strong  Law  of  Large  Numbers 


[  lim  - ,  lim  - ] 

^o-oo  Ao,i(no)  Ai,i(ni) 


=  [  lim 


,no  -  1  1 


lim 


,ni  —  1  1 


no^oo  no  Ao,i(no)  Ai,i(ni)' 

r  ^  1  1  ' 

=  a.s. 


(25) 


From  probability  theory  we  know  that  If 


lim  Xr,  =  C  a.s. 


(26) 


where  C  >  0,  then 


1  1 

lim  ^ 

n^oo  Jin  O 


(27) 


Since  all  \j/s  are  strictly  positive,  by  choosing  consistency  follows  from  (27). 


D.  Asymptotic  Statistics  Properties:  Asymptotic  Distribution 


Perhaps  the  most  common  distribution  to  arise  as  an  asymptotic  distribution  is  the  normal 
distribution.  In  particular,  the  central  limit  theorem  provides  an  example  where  the  asymptotic 
distribution  is  the  normal  distribution. 

Consider  the  reciprocal  of  the  estimators  in  (12)  and  (13) 

^  (28) 
Ai,*(n)  n-\  n 

where  are  i.i.d  with  exponential  distribution  and  parameter  Aj,*.  From  CLT  (Central  Limit 
Theorem)  we  have 


In  general  if  Yn 
is 


n  —  1  ^  1 

Aj,i(n) 


4Ar(o, 


(29) 


the  distribution  of  the  reciprocal  of  the  non-negative  random  variable 


FyM  =  Pr{Fn  <y}  =  Pr{^  <y}  =  Pr{X„  >-}  =  !-  Fx„(-),  (30) 

Xn  y  y 

and  for  the  pdf  we  have 

fvjy)  =  -fxj-)-  (31) 

y  y 

Therefore  since  -  ^  has  asymptotic  distribution  given  in  (29),  the  asymptotic  distribution  of 
the  estimator  \j,i{n)  can  be  obtained  in  closed-form  by  (30),  and  (31). 
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E.  Asymptotic  Statistics  Properties:  Asymptotic  Efficiency 

We  now  establish  the  asymptotic  efficiency  of  SEINE  for  the  class  of  sparse  sensing  policies. 
The  reason  to  consider  only  sparse  sensing  policies  is  that,  in  practice,  when  switching  cost  is 
not  negligible,  each  channel  should  be  observed  at  least  for  some  minimum  amount  of  time,  and 
if  there  are  many  channels  to  consider,  the  gaps  between  observation  intervals  are  sufficiently 
large.  Recall  that  a  sensing  policy  tt  is  defined  by  the  allocation  of  sensing  time  T'^  =  {Tfj} 
where  Tf-  is  the  duration  of  the  jth  sensing  interval  on  channel  i.  We  denote  the  total  sensing 
time  by  T  =  \T'^\  =  Tf.  The  MSE  of  a  channel  estimator  associated  with  sensing  policy  tt 
is  given  by 

M 

MSE(T")  =  5^{(Ao,*  -  Ao,*)'  +  (Ai,i  -  Ai,,)^}.  (32) 

i=l 

Eor  the  class  of  sparse  sensing  policies  (as  defined  in  Theorem  2),  Theorem  2  shows  that  the 
Eisher  information  /(T^)  associated  with  is  upper  bounded  by  the  Eisher  information  when 
each  channel  is  allocated  with  a  single  sensing  interval 


/(T")  </i(T"), 


(33) 


where  is  the  Eisher  information  when  each  channel  is  allocated  with  a  single  sensing 

interval  with  total  sensing  time  |T^|.  Therefore,  for  a  sparse  sensing  policy  tt,  the  optimal  CRB 
is  obtained  under  the  class  of  policies  Hi  (with  single  installment  per  channel)  in  (18).  Hence 
we  define  the  asymptotic  efficiency  by 


77^  =  lim 
IT- He 


CRB*(|r^|) 


(34) 


MSE(r^)  ’ 

where  CRB*(|T’"|)  =  min^gm  CRB(T^)  defined  in  (18)  is  the  optimal  CRB  for  the  class  of 
policies  Hi  which  is  optimal  for  the  class  of  sparse  policies  as  well  based  on  (33). 

When  the  sensing  policy  is  randomized,  the  resulting  allocation  of  sensing  time  T'^  is  also 
random.  We  use  stopping  times  (the  time  instants  when  a  certain  number  of  complete  idle  and 
busy  periods  are  at  hand)  in  the  calculation  of  MSE  to  deal  with  the  randomness,  and  we  define 
the  relative  efficiency  the  same  way  as  (34).  Note,  however,  that  the  relative  efficiency  is  also 
random.  A  sensing  and  estimation  policy  tt  is  almost  surely  asymptotically  efficient  if 


lim  =  1 

[T’^Hoo  MSE(T’") 


a.s. 


(35) 
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The  following  theorem  establishes  the  a.s.  asymptotie  effieieney. 

Theorem  4:  The  asymptotie  effieieney  of  the  Poliey  SEINE  eonverges  to  1  asymptotieally 


almost  surely.  Namely 

^SEINE 


lim 

|7^SEINEj^QQ 


CRB*(|T^e™E|^ 

MSE(rsEiNE) 


1  a.s. 


(36) 


Proof:  See  Appendix  C. 


V.  Simulation  Results 

In  this  seetion  the  simulation  results  are  presented  in  three  subseetions.  Eirst  we  present  the 
simulation  results  showing  the  eonsisteney  of  the  estimator,  then  we  present  the  simulation  results 
regarding  the  asymptotie  distribution  of  the  estimator,  and  at  the  end  we  show  the  asymptotie 
eonvergence  of  the  MSE  of  poliey  SEINE  to  the  Cramer-Rao  bound  (asymptotie  eonvergenee 
of  effieieney  to  one).  Eor  simulation  a  network  eonsisting  of  4  ehannels  oeeupied  by  primary 
system  is  eonsidered  with  the  following  parameter  set 

=  [0.1,  0.6,  0.2,  0.5], 
a;  =  [0.7,  0.1,  0.8,  0.9]. 

The  sequenee  of  round  lengths  used  in  the  poliey  SEINE,  {cn}^=Q,  is  pieked  to  be  exponentially 
inereasing  with  n.  The  poliey  SEINE  is  run  for  22  rounds  for  200  monte-earlos.  We  refer  to  this 
setting  as  Case  1. 


A.  Consistency 

In  this  seetion  we  illustrate  the  eonsisteney  of  our  estimator  by  simulation.  In  Eig.  5  the 
estimated  Aq  and  Ai  (averaged  over  all  monte-earlo  runs)  for  all  ehannels  are  presented.  It  is 
shown  that  all  of  the  estimated  parameters  eonverge  to  their  true  values. 

In  Eig.  6  two  parameters  Ao,i  (Aq  for  channel  1),  and  Ao,3  (Aq  for  channel  3)  are  chosen 
and  the  estimated  sequence  is  plotted  for  all  the  Monte-Carlo  runs.  As  shown  in  the  figure  the 
estimated  sequence  converges  to  the  true  values  for  all  the  Monte-Carlos  verifying  the  almost 
surely  convergence  (strong  consistency). 
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Fig.  5.  Convergence  of  estimated  parameters  for  all  channels 


Convergence  for  all  monte-carlo  runs  (Consistency) 


Fig.  6.  Convergence  of  estimated  parameters  for  all  monte-carlos 
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n=19 


n=20 


n=21 


Fig.  7.  Histogram  of  the  Ai,i(n) 


B.  Asymptotic  Distribution 

In  this  section  we  plot  the  histogram  of  the  estimated  sequence  for  Ai^  (Ai  for  channel  1).  Fig. 
7  shows  the  distribution  of  the  Ai  i(n)  for  different  round  indexes  n,  and  how  it  is  distributed 
around  and  converges  to  Ai^  =  0.7  as  the  round  index  n  grows. 

C.  Asymptotic  Efficiency 

In  this  section  we  plot  the  MSB  of  the  policy  SEINE  vs.  the  optimal  Cramer-Rao  bound  for 
the  sparse  policies  (CRB*).  Here  we  consider  another  network  setting  with  the  same  number  of 
channels  but  different  parameter  set 


Ao  =  [0.3,  0.2,  0.3,  0.4], 
a7  =  [0.3,  0.3,  0.2,  0.15]. 
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Fig.  8.  MSB  vs.  CRB*:  Case  1 


Time 


Fig.  9.  MSB  vs.  CRB*  :  Case  2 
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The  policy  SEINE  is  run  the  same  as  the  case  1  for  22  rounds  with  200  monte-carlo  runs.  We 
refer  to  this  new  setting  as  case  2.  The  MSE  of  policy  SEINE  vs.  CRB*  is  plotted  for  both  case 
1,  and  case  2  in  Eig.  8  and  Eig.  9.  As  you  can  see  the  MSE  asymptotically  converges  to  CRB* 
indicating  the  asymptotic  efficiency  of  the  policy  SEINE  for  the  estimator  within  sparse  policies. 

VI.  Conclusion 

In  this  paper,  we  have  considered  the  problem  of  multichannel  estimation  in  OSA.  The  channel 
occupations  of  the  primary  system  are  modeled  by  two-states  Markov  processes.  Secondary 
system’s  objective  is  to  learn  the  channel  parameters  in  the  optimal  way  in  order  to  maximize 
the  estimator’s  efficiency.  A  sequential  policy  is  proposed  that  achieves  the  optimal  efficiency 
asymptotically. 


Appendix  A:  Proof  of  Theorem  1 


Erom  (5) 


where 


VLA.i(V,A,)  = 


”-0-|-l{s=l} 


1 


1 


(Ao+Ai)2  (Ao-l-Ai)2 

1  ^1-H{s=0}  1 


(Ao-l-Ai)2 


Af 


(Ao+Ai)2 


we  get 


Ao -l- Ai  A^ '  ^  Ao -l- Ai  A^_g  '  ^  (Aq-I-Ai)^ 

no-c°,  ^  __  . _ T  ^  Ai 


=  Pr{n  is  even} 


-I-  Pr{n  is  even} 


Ao  -I-  A: 


-E{- 


A? 


Ao  r  ^0  -  Ce 


Ao  -I-  A; 


-E{- 


Af-. 


|s  =  0,  n  is  even}  -|-  Pr{n  is  odd} 


|s  =  1,  n  is  even} 


Ao  -I-  A; 


-E{^°  \s  =  0,n  is  odd} 
Ac 


-I-  Pr{n  is  odd}  ,  — —\s  =  I,n  is  odd}  — 


Ao  -I-  Ai 


AL« 


(Ao  +  Ai)2 

AiCq  -I-  Aoc}  , 


=  E{^}  -I-  Pr{n  is  even}(^^^'"  +  Pr{n  is  odd}( 

Aq  Ao  -I-  Ai  Ao  -I-  Ai 

=  IE{^}-|-/o, 


f  ID  I  •  1 +  -^oCe,  I  -n  r  •  ' 

Jo  =  Pr|n  IS  even}( — - - - - )  -f  Prjn  is  odd}(- 


(37) 


Ao  +  Ai  ^  ^  Ao  +  Ai  ^ 

is  a  bounded  function  of  parameters  Aq,  Ai,  and  Ti,  and  c^,  and  are  some  constants  depending 
on  n  being  even  or  odd  and  also  s;  but  since  they  are  constants,  their  exact  value  does  not  affect 
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our  final  results.  Similarly  we  get 


And  clearly  for  the  cross  terms  E{ 


92  L 


}  = 


dXidXo 

E{no}  =  E{ni}  =  - 


(Ao+Ai)2 

T 


.  Note  that 


Ao  Ai 


(38) 


Therefore 


[if, 


X2(  W  1  N  +  /o  — 


(Ao+Ai)2 


(Ao+Ai)2 


{Ao+Ai)2 

From  definition  of  /o,  and  /i  we  can  get 


+  /i 


(Ao+Ai)2 


■  [^1 1  ti  -\-T] 


+  o(Ei)) 


^olxT+tr) 


{Ao+Ai)2 


(Ao+Ai)2  A2(^  +  J-) 


jT+j_i(3  +  0(^1)) 

Ao^Ai' 


Appendix  B:  Proof  of  Theorem  2 

Let  first  consider  the  Likelihood  function  for  the  case  we  have  two  disjoint  intervals.  Assume 
that  two  intervals  +  Ti]  U  [12,^2  +  ^"2]  are  observed.  Therefore  the  likelihood  function 
becomes 

=  /(-Si,  {Zh,l)hLl\^0,  Ai)/(s2,  {Zh,2)hLl\^0,  Ai,  Sti+Ti  =  s') 

=  f{su  {zh,i)T=Mo,  Ai)  Pr{^,,  =  92|^t,+T,  =  s'}f{{z^,2)lU\Xo,  Ai,  =  s^) 


—  f{si,{Zh,l)hLl\^0,^l) 


PrjS't,  =  S2\St,+Ti  =  g'} 
Pr{S't2  =  'S2} 


f{s2,  {Zh,2)hLl\^0,  Al 


r  Pr{>St,  =  g2|>^fi+Ti  =  s'}  ^ 
J'^1  r  o  _  „  1  • 


Pr{S't2  =  S2} 

Here  we  denote  the  likelihood  function  of  a  single  interval  of  length  Tj  as  obtained  in  previous 
section  by  /j^  .  It  is  seen  that  the  extra  information  provided  by  the  discrete  sampling  is 

Pr{^t2  =  Ssl^ti+Ti  =  s'} 

Pr{^t.  =  S2}  •  ^  ^ 

It  is  apparent  that  when  these  intervals  are  far  enough  so  that  St2  and  St^+Xi  are  independent, 
this  term  becomes 

Pr{S,,  —  S2\Sti+Ti  —  s'}  Pr{S't2  =  S2} 


Prj^t^  =  S2} 


Pr{S't2  =  S2} 


=  1. 


(40) 
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Thus  the  discrete  sampling  part  will  not  provide  any  extra  information  and  the  Fisher  information 
will  be  the  sum  of  the  Fisher  information  for  these  two  intervals. 

The  Fisher  information  matrix  for  [fi,  ti  +  Ti]  U  [^2,  ^2  +  T2\  is 


^[tl,tl+Tl]U[t2,t2+T2]  —  +  I[tl,tl+T2]  +  ^\ti,ti+Ti]U[t2,t2+T2]^ 


(41) 


where 


tD  =¥(-V^  1  —  'S2|*S'ij+ri  —  -s'} 

Hti,ti+Ti]u[t2,t2+T2]  -  H-^Xo,Xi  iogl  ^^777^  77  ))■ 


(42) 


Pr{S't2  =  ^2} 

Now  based  on  the  above  argument  and  (5),  we  compare  the  Fisher  information  matrixes  for 
these  two  cases  of  single  and  two  intervals. 

For  the  single  interval  [to,  to  +  T],  the  Fisher  information  matrix  is 

»o+l{so=i}  1  _ 7 


'-[to,  to+T] 


=  E 


-'o 


(Ao+Ai)2  (Ao+Ai)2 

1 _  ^l+l{so=0}  _ 1 


(Ao+Ai)2 


Af 


(Ao+Ai)2 


The  Fisher  information  matrix  for  [ti,ti  +  Ti]  U  [t2,  ^2  +  T2]  is 


I[tl,tl+Tl]U[t2,t2+T2] 


htuti+Ti]  +  I[ti,ti+T2] 


-J- 

'  [tl,tl+Ti]U[t2,t2+T2]  • 


(43) 


It  can  be  rewritten  as 

^tl,tl+Tl]U[t2,t2+T2] 


=  E 


E 


=  E 


'^0  +  ^{ai=l}  _ 1 


(Ao+Ai)2 


(Ao+Ai)2 


{Ao+Ai)2 


"•l+l{si=0} 


(Ao+Ai)2 


»0  +  l{s2  =  l} 


-'o 


(Ao+Ai)2 


(Ao+Ai)2 


(Ao+Ai)2 


'‘'0+”'0  +  J-{3i=l} +1(82  =  1} 


11  +  1(82=0} 


(Ao+Ai)2 


(Ao+Ai)2 


(44) 


'  '^[tl,tl+Tl]U[t2,t2+T2] 


(Ao+Ai)0 


(Ao+Ai)2 

_  '} _ 

A^  (Ao+Ai)2 


nj +”-1+1(81=0} +1(82=0} 


rD 


[t  1 ,( 1  +Ti  ]  U  [(2  ,(2  +T2]  ’ 


where  j  =  1,  2,  and  s  =  0, 1  is  the  corresponding  value  of  Ug  for  the  jth  interval.  Note  that 
since  chopping  the  interval  [to,  to  +  T]  into  two  pieces  gives  smaller  number  of  complete  idle 
and  busy  periods  in  expectation,  we  have 


E[n^  +  +  l{si=i-s}  +  l{s2=i-s}]  ^  E[ns  +  l7p=i_5}  . 


(45) 
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Using  this  inequality  we  get 


f  [^l,^l+Tl]U[^21^2+^2] 


=  E 


'‘'0+^0  +  l{ai=l}+l{32  =  l} 


(Ao+Ai)2 


(Ao+Ai)2  (Ao+Ai)2 

n}+n^+l{si^o}+l{a^^o} 


(Ao+Ai)2 


^  [tlM+Tl\0[t2,t2+T2\ 


<  E 


^0  +  1(30^1} 


^0 


(Ao+Ai)2  (Ao+Ai)2 

2  ^i+l{3o=o} _ ^ 


=  h 


(Ao+Ai)2 

1 


(Ao+Ai)2 


'  [tl,tl+Ti]  U[t2  5^2+72] 


[iOi  ^0+"^] 


(Ao  +  Ai 


1  1 
1  1 


tD 


^  [tl,tl+Ti]U[i2,^2+72]  * 


From  (40)  it  is  clear  that  when  Ati  — oo  the  discrete  sampling  gives  no  extra  information. 
Therefore 

r 

0  0 


Atj^oo  hiiM+Ti\iJ[t2,t2+T2\ 


Also  note  that  the  matrix  1  is  positive  definite 

1  1 


0  0 


(46) 


1  1 


>  0. 


(47) 


Therefore  when  Afi  — >  oo  (which  makes  the  matrix  ti+Ti]u[t2  t2+T2\  we  have 


^'[tl,ti+Ti\\j[t2,t2+T2\  A  to+T] 


(Ao  +  Ai)2 


1  1 
1  1 


<  /[ 


[to,  to+T]- 


(48) 


Thus  from  (46)  we  know  that  3r  >  0  such  that 
r  =  inf{Ati  >  0  | 

For  every  Ati  >  r 

1 

7[ti,ti+Ti]U[t2,t2+T2]  A  to+T]  /  ■,  I  \  \2 

(Ao  +  M) 


< 


tl+Tl]U[t2,t2+T2]  -  ^  ^^^2 


1  1 
1  1 


}• 


(49) 


1  1 
1  1 


+  -^]?i,ti+Ti]u]t2,t2+T2]  ^  ho,  to+T]-  (50) 


It  means  that  sensing  one  continuous  interval  has  higher  Fisher  information  than  chopping  it 
into  two  pieces. 

For  K  intervals  with  minjA)^{Atj  }  >  r  we  can  use  induction.  We  take  the  first  two  intervals 
and  replace  them  by  one  interval  to  get  K  —  1  intervals.  We  do  this  procedure  for  —  1  steps 
to  get  a  single  interval  with  length  T  =  J2f=i  h- 
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Appendix  C:  Proof  of  Theorem  4 

Since  the  MSB  is  obtained  under  the  policy  SEINE,  in  what  follows  we  drop  the  superscript 
SEINE  for  simplicity. 

Lemma  3: 


lim  ai{n)  =  Jim  df  =  o;*,  Vi  G  {1, ,  M}  a.s.  (51) 

n  ^oo 

where  k  =  ai{n)  is  the  estimated  ctj  at  the  end  of  round  n,  and  df  is  the  estimated  ctj 

when  k  complete  samples  of  idle  and  busy  periods  is  at  hand  for  all  channels. 

Proof:  limg^oo  df  =  ctj  is  based  on  the  strong  Eaw  of  Barge  Numbers 

lim  (Ao,i,  Ai,i)  ^  (Ao,i,  Ai,i)  a.s.  Vie  {!,...  M},  (52) 

and  the  fact  that  at  is  a  continuous  function  of  (Aqj-,  Aij),  j  e  {1, . . .  M}. 
lim„^oo d;i(n)  =  a.s.  is  a  direct  consequence  of  lemma  2.  (lim„^oo d;i(n)  = 

on  the  event  i.o.}  and  this  event  has  probability  1).  ■ 

Define  the  stopping  times 

Tfc.  =  inf  {time  |  exactly  ki  complete  idle  and  busy  periods  is  sensed  on  channel  i}.  (53) 


Consider  the  time  sequences  MSEJ  is  the  sum  of  MSEs  for  the  parameters  Aq*  and 

Ai^j  at  the  time  instant  It  is  computed  for  the  estimators  in  (12),  and  (13)  using  the  complete 
idle  or  busy  periods  of  channel  i  which  can  be  rewritten  as  Xjfki)  =  where  zfs  are 

i.i.d  exponential  random  variables  with  parameter  Aj  j. 

In  general  if  X  =  where  piS  are  i.i.d  exponentially  distributed  random  variables  with 
parameter  A,  X  has  Erlang  distribution  of  degree  n  with  parameter  (n  —  1)A.  Therefore  for  the 
MSB  of  4  we  have 


MSB  = 


E{(^  -  A)"}  =  /  -  27  +  A2)A^ - - dt 


IQ 


'E  t 


=  A 


2  /  ((n- ((n-l)A) 


(n  —  1)! 

2  /■“  ((n  -  l)y)n-2^n-3g-A(n-l)t 


'0 


(n  —  1)! 


(n-  l)(n-2)  Jo 


(n  —  3)! 


dt 


((n— 1)A)  — 

— - : -  /  - - “T": - dt 


n  —  1 


(n-2)! 


A^ 


n  —  2 
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Therefore  for  our  problem  we  get 

MSEi 

Denote 


ki-2 


(54) 


T(n) 


Ti(n) 


T’in) 


Tiin.) 

T‘{n,) 


=  total  time  spent  on  the  system  at  the  end  of  round  n. 
=  total  time  spent  on  channel  i  at  the  end  of  round  n. 

=  total  time  spent  on  channel  i  at  the  end  of  round  n 

that  only  includes  complete  idle/busy  periods. 

=  total  time  spent  on  channel  i  up  to  time 

=  total  time  spent  on  channel  i  up  to  time  that 

only  includes  complete  idle/busy  periods. 


Lemma  4: 


lim 

ki—^OO 


THn.) 

Ti(n,) 


a.s. 


Proof:  Note  that  Tfrkfj,  and  T*{Tkfj  differ  only  at  the  incomplete  idle/busy  periods  that 


only  happens  at  the  beginning  and  the  end  of  each  round.  Therefore 


^  X]  Periods 

TfTkf)  TfTkf) 


Denote 


A 


n*= 


sup{n  :  Tiin)  <  Ti(rfcJ}, 


then  the  number  of  incomplete  periods  is  at  most  2n*  +  1.  Thus 

^  incomplete  periods  ^  incomplete  periods 


Note  that 


0  <  lim 

n*— ^oo 


=  lim 

n*— ^oo 


TfTkfj  Tfnfj 

Y  incomplete  periods 


{{mf  +  o(m^))^  +  (m*  +  o(m*))^) 

Y.7=iCjafj) 


(2(n* +  o(n*))  +  1):^ 

<  lini  - — 

-  — 


a.s. 
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where  w}  +  rrv  =  2n*  +  1,  and,l/7j  =  max{l/Ao,i,  l/Ai,*}.  Sinee  ctj  >  0, 

3ei  >  Os.t.  cti  —  e*  >  0. 


From  lemma  3, 


3n^^s.t.  'in  >  n^^,  \ai{n)  —  ai\  <  e*. 


Therefore 

n*  n* 

i=i  i=i 

where  a™  =  min{minj=i^...„^,{dj(j)},  —  e*}  >  0.  Thus 

^  ineomplete  periods 

Tiini) 

(2(n*  +  o(n*))  +  1):^ 


0  <  lim 

n*—^co 


<  lim  .  .s 

n*^oo  2^,=lCjai(j) 


7i 


(2(n*  +  o(n*))  +  l)^ 
<  hm  -  ' 

n*—^co 


By  definition  lim„^^oo  ^  =  cxd,  therefore 


Q  ^  j.  X]  incomplete  periods  ^  ^ 

“  n*^oo  Ti{n^)  ~ 


Then  the  lemma  follows. 
Lemma  5: 


lim 


Ti{rki] 


1  1 
+ 


fci— >00  ki  Aq  j  Al  j 

Proof:  From  lemma  4  we  have 


a.s. 


r  TfTkf)  T*{Tki) 

hm  — =  lim  — —  a.s. 


ki^oo  ki  ki^oo  ki 


The  equality 


lim 


T:{Tk:^ 


1  1 
+ 


a.s. 


ki^oo  ki  Ao,i  Ai^j 

is  based  on  the  strong  Law  of  large  numbers  since  T*{Tki)  is  exactly  equal  to  sum  of  ki  complete 
idle,  and  busy  realizations  divided  by  ki.  ■ 
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We  have 


therefore  from  Lemma  5  we  get 


=  fe'' 


hm  T,(r,jMSE;^  =  +  A2,)(—  +  — ) 

ki^oo  *  Ao,i  Ai,i 


a.s. 


Note  that 


Also 


Vn  eN,3k  =  s.t.  <  T,(n)  < 


Jim  Tiirj^jMSEl  <  lim  Ti(n)MSE;  <  Jim  JMSEJ 

fc^oo  ^  fc^oo  ‘ 


MSE*  (T(n))  =  MSE^  =  MSEJ,  a.s. 


Therefore  we  have 


lim  r,(n)MSE'(r(n))  =  +  A;J(—  +  — )  o.s. 

Ao,i  Al^i 


Lemma  6: 


^.(n) 

hm  ,  =  ttj  a.s. 


T(n) 


Proof:  Erom  lemma  3  we  have 


lim  cniin)  =  a.s. 


Therefore 


Ve  >  0,  3ni  s.t.  Vn  >  ni,  |Q;j(n)  —  ai|  <  -  a.s. 


Eor  this  given  ni,  3n2  such  that  Vn  >  n2, 

Eni 

fc=iCfc  ^  e 


<  -. 


Y2=,Ck  2- 

Then  Vn  >  n2, 

_  ELiCfctt*(/^)  _  EfciiCfc(ai(/^)  -  tti)  ,  EEiCfctti  +  ELni+iCfcai(/i^) 


ELiC. 


En 

k=l 


En 

k=l 


(55) 


(56) 
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Since  0  <  ai{k)  <  1,  and  0  <  <  1, 


e 

“2^ 


YlUc,  ,  St, ct  ^  £ 

— ^r)  ^  • 


En 

k=l 


ELiCk  2- 


(57) 


Also 


En  I  v^Ti  ^  /  /  \  \~^n 

k=ni+l^^  ^  ^  Z^fc=ni+1 ^  I  ^  l^k=ni+l^^  ^  ^ 

o  v-^n  v-^n  ^  v^n  ^  <^*5. 

2  2  Ck  ^k=i  ^k  2  Cfc  2 

Combining  it  with  (56),  and  (57),  we  get  Ve  >  0,  3n2,  s.t.  Vn  >  n2, 

T.(n) 


T(n) 


<C  0^2  “(“  6.  Cl. 5. 


Thus 


y  T,{n) 
lim  =  on 

n^oD  1  [n) 


a.s. 


From  lemma  6  and  (55)  we  get 


(^0,i  +  ^li)(^+  A)-) 


lim  T(n)MSE*(T(n))  = 


a.s. 


Oli 


Therefore 


M 


lim  T(n)  VMSE*(T(n))  =  V 

71. — ^rv~)  f  ^  f  ^ 


"  w,.  +  H.)(i  +  A) 


a.s.. 


2=1 


2=1 


Oii 


or  equivalently  for  the  efficiency 


,.  CRB*(T)  ^ 

hm  - - —  =  1  a.s. 

T^oo  MSE(T) 
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